Supplementary Material: Linear Convergence of Stochastic Frank Wolfe Variants
نویسندگان
چکیده
Proof. Consider the function class F = {f(·,x) | x ∈ P} as defined in (SP1), that is f(i,x) = fi(x). Since fi(·) each is assumed to be Lipschitz continuous with Lipschitz constant Li, we must have |fi(x)− fi(y)| ≤ LF ‖x− y‖, where LF ≡ max{L1, . . . , Ln}. Moreover, the index set P ∈ R for the function class F is assume to be bounded. Therefore all conditions for Lemma 2 are satisfied and hence the number of brackets of the type [f(·,x)− LF , f(·,x) + LF ] satisfies
منابع مشابه
Linear Convergence of Stochastic Frank Wolfe Variants
In this paper, we show that the Away-step Stochastic Frank-Wolfe (ASFW) and Pairwise Stochastic Frank-Wolfe (PSFW) algorithms converge linearly in expectation. We also show that if an algorithm convergences linearly in expectation then it converges linearly almost surely. In order to prove these results, we develop a novel proof technique based on concepts of empirical processes and concentrati...
متن کاملLinear Convergence of a Frank-Wolfe Type Algorithm over Trace-Norm Balls
We propose a rank-k variant of the classical Frank-Wolfe algorithm to solve convex optimization over a trace-norm ball. Our algorithm replaces the top singular-vector computation (1-SVD) in Frank-Wolfe with a top-k singular-vector computation (k-SVD), which can be done by repeatedly applying 1-SVD k times. Our algorithm has a linear convergence rate when the objective function is smooth and str...
متن کاملNew Active-set Frank-wolfe Variants for Minimization over the Simplex and the `1-ball
In this paper, we describe a new active-set algorithmic framework for minimizing a function over the simplex. The method is quite general and encompasses different active-set Frank-Wolfe variants. In particular, we analyze convergence (when using Armijo line search in the calculation of the stepsize) for the active-set versions of standard Frank-Wolfe, away-step Frank-Wolfe and pairwise Frank-W...
متن کاملOn the Global Linear Convergence of Frank-Wolfe Optimization Variants
The Frank-Wolfe (FW) optimization algorithm has lately re-gained popularity thanks in particular to its ability to nicely handle the structured constraints appearing in machine learning applications. However, its convergence rate is known to be slow (sublinear) when the solution lies at the boundary. A simple lessknown fix is to add the possibility to take ‘away steps’ during optimization, an o...
متن کاملFrank-Wolfe with Subsampling Oracle
We analyze two novel randomized variants of the Frank-Wolfe (FW) or conditional gradient algorithm. While classical FW algorithms require solving a linear minimization problem over the domain at each iteration, the proposedmethod only requires to solve a linear minimization problem over a small subset of the original domain. The first algorithm that we propose is a randomized variant of the ori...
متن کامل